• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The present application discloses a method and apparatus for detecting a vehicle contour based on point cloud data. The method includes: acquiring to-be-trained point cloud data; generating label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud, the labeling used to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour; training a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and acquiring to-be-detected point cloud data, and obtaining a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model. The implementation may achieve an accurate detection of the vehicle contour.
    公开号:SE1651131A1
    申请号:SE1651131
    申请日:2016-08-24
    公开日:2017-07-28
    发明作者:Li Bo；Zhang Tianlei；Xia Tian；Tao Ji；Lin Yuanqing
    申请人:Baidu Online Network Tech (Beijing) Co Ltd；
    IPC主号:
    专利说明:

    METHOD AND APPARATUS FOR DETECTING VEHICLE CONTOUR BASED ONPOINT CLOUD DATA CROSS-REFERENCE TO RELATED APPLICATION This application is related to and.claims priority froHlChineseApplication Ser. No. 201610057306.X, filed on January 27, 2016,entitled "METHOD AND APPARATUS FOR DETECTING VEHICLE CONTOURBASED ON POINT CLOUD DATA", the entire disclosure of Which is hereby incorporated by reference.TECHNICAL FIELD The present application relates to the field of computertechnology, specifically to the field of vehicle detectiontechnology, and more specifically to a method and apparatus for detecting a vehicle contour based on point cloud data.BACKGROUND With the economic:development and living standards improvement,the vehicle, as an indispensable commuting tool, has graduallyentered into millions of households. However, the associateddriving safety problem brings cloud into the sunny happy life.The number of deaths in traffic accidents in year 2010 in Chinais 65,225 as reported by the Ministry of Public Security, andis 275,983 as estimated by the WHO model, which is equivalentto a major air disaster every day. The number of casualtiescaused by traffic accidents in the world is equivalent to 13 times of that of the 911 disaster every year.
    In the modern society with advanced science and technology,the populace clearly cannot tolerate successive occurrencesof quasi-disasters. The need to drive safely has beenunanimously' recognized. by the society. As driven. by thegovernment and the market, the vehicle safety technology hasstarted to prosper. In terms of general classification, the vehicle safety technology may be divided into two main 1 / 42 lO categories, namely, the passive safety and active safety. Thepassive safety refers to the technologyfto reduce the loss aftera collision to a minimum, mainly the design related to thevehicle structure, seat belt, laminated windscreen glass,collapsible steering column, airbag, etc. However, since themechanism of the passive protection is not triggered until thecollision.happens, the number of accidents which may be avoidedis limited. The active safety refers to the advance warningtechnology to avoid the collision, which mainly relates to theperception, understanding, decision, and control on thedriver's surrounding environment, such as the lane departurewarning (LDW). In comparison, the active safety, for itsadvantages such as the foresight and the weak association withthe vehicle design, has been widely studied, and debuted as the products of the Advanced Driver Assistance System (ADAS).
    Vehicle detection technology is one of the essential technicalpillars for the active safety. The existing technical solutionsmostly adopt a single visual technology, which detects thevehicle by using a method of matching the features in imagesand is not capable of accurately providing the spatial locationand attitude of the vehicle to be detected. Other technicalsolutions exist, which use laser point clouds as inputs andcategorize the features such as the shape and point clouddensity' before detecting' the vehicle. This method Inostlyrequires significant manual adjustments, and. has a poor adaptability to scenario changes.
    SUMMARY OF THE INVENTION An objective of the present application is to propose animproved method and apparatus for detecting a vehicle contourbased on point cloud data, so as to solve the technical problems mentioned in the Background.
    In the first aspect, the present application provides a methodfor detecting a vehicle contour based on point cloud data, comprising: acquiring to-be-trained point cloud data; 2 / 42 lO generating label data corresponding to points in theto-be-trained point cloud data in response to labeling on thepoints in the to-be-trained point cloud, the labeling used toindicate whether each of the points in the to-be-trained pointcloud data is on a vehicle contour; training a fullyconvolutional neural network model based on the points in theto-be-trained point cloud data and the label data correspondingto the points in the to-be-trained point cloud data, to obtaina vehicle detection model; and acquiring to-be-detected pointcloud data, and obtaining a detection result corresponding toeach to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model.
    In some embodiments, the acquiring of to-be-traiïmxípoint clouddata comprises: mapping to-be-trained three-dimensional data,collected.by a point cloud data collecting device, to generateato-be-traineddepthmap,thevaluesofrespectivepixelpointsin the to-be-trained depth map being respective points in theto-be-trained point cloud data corresponding to the to-be-trained three-dimensional data.
    In some embodiments, the to-be-trained three-dimensional dataD (x, y, z) is a coordinate in a coordinate system taking thepoint cloud data collecting device as an origin, an upwardvertical direction as a z axis, and any two axes perpendicularto each other in a horizontal plane as an x axis and a y axis;the coordinate of each pixel point in the to-be-trained depth map is (r, c), and the value of each of the pixel points is (d, z); wherein:ßw/Aejßlø/Aøld== x2+3fi 6 = atan2(y,x) d) = arcsin(z/W/x2 + yz + ZZ) G is an angle between a line connecting the to-be-trained 3/42 lO three-dimensional data D to the origin and a first plane, A6is a horizontal resolution of the point cloud data collecting device, and the first plane is a plane where the y axis andthe z axis locate; and $ is an angle between a line connectingthe to-be-trained three-dimensional data D to the origin anda second.plane, Ao is a vertical resolution of the point clouddata collecting device, and the second plane is a plane where the x axis and the y axis locate.
    In some embodiments, the generating of label data correspondingto points in the to-be-trained point cloud data in responseto labeling on the points in the to-be-trained point cloud datacomprises: transforming a point in the to-be-trained pointcloud data to generate two-dimensional label data with 26channels if the point is on the vehicle contour, wherein thevalue of a first channel in the 26 channels is l, the valueof a second channel in the 26 channels is O, and the 3“í- 24Üchannels in the 26 channels are respective rotation vectorsof coordinates of vertexes of a three-dimensional cube wherea vehicle is located with respect to a three-dimensionalcoordinate of the point on the vehicle contour in theto-be-trained point cloud data; and transforming a point inthe to-be-trained point cloud data to generate two-dimensionallabel data with 2 channels if the point is not on the vehiclecontour, wherein the value of a first channel in the 2 channelsis O, and the value of a second channel in the 2 channels is l.
    In some embodiments, the rotation vector (x“p, y'p, z'p) of acoordinate of a vertexe of a three-dimensional cube where avehicle is located with respect to a three-dimensionalcoordinate of the point on the vehicle contour in the to-be-trained point cloud data is: 4/42 lO wherein R is a rotation matrix of a point P(xfl jq, zi) on the vehicle contour in the point cloud data, and:R I Rzzæzfleyi . 61. = atan2(yl. ,xl.) Q=am¶mj/¶xf+yf+zf)_ _cos6, -sü16, 0Rá(6¿)= sü16, cos6, 00 0 1 _cos$, 0 sü1QfRyl-(Öf) I 0 1 0 _-sü1$, 0 cos$¿_ In some embodiments, the training of a fully convolutionalneural network model based on the points in the to-be-trainedpoint cloud data and the label data corresponding to the pointsin the to-be-trained point cloud data, to obtain a vehicledetection1nodel comprises: initializing parameters of the fullyconvolutional neural network; adjusting the parameters of thefully convolutional neural network based on a loss function,wherein the loss function is a deviation between the label datacorresponding to a current output and the label datacorresponding to the to-be-trained point cloud data, and thecurrent output is an output of the fully convolutional neuralnetworkcorrespondingixncurrentto-be-trainedpointclouddataand. current parameters of the fully' convolutional neuralnetwork; outputting' parameters corresponding' to the loss function as parameters of the vehicle detection model if the 5/42 lO loss function is at a minimum value; and otherwise, returningto execute the step of adjusting the parameters of the fully convolutional neural network based on a loss function.
    In some embodiments, the acquiring of to-be-detected pointcloud data, and obtaining of a detection result correspondingto each to-be-detected point in the to-be-detected point clouddata based on the vehicle detection model comprises: mappingto-be-detected three-dimensional data, which is collected bythe point cloud data collecting device, to generate ato-be-detected depth map, the values of pixel points in theto-be-detected. depth. map corresponding' to respectiveto-be-detected points in the to-be-detected point cloud datacorresponding to the to-be-detected three-dimensional data;acquiring a predicted result of each of the to-be-detectedpoints in the to-be-detected point cloud data based on thevehicle threshold.
    In some embodiments, the predicted result further compriseslocation information of the to-be-detectecípoint; the acquiringofto-be-detectedpointclouddata,andobtainingofzadetectionresult corresponding to each to-be-detected point in theto-be-detected point cloud data based on the vehicle detectionmodel further comprises: acquiring first to-be-detected pointswhose probabilities are each greater than the preset thresholdin the to-be-detected point cloud data, to form a set of thefirst to-be-detected points; and generating spatial vehiclelocation information corresponding to the to-be-detected pointcloud data based on the location information of theto-be-detected points in the set of the first to-be-detected points. 6/42 lO In the second aspect, the present application provides anapparatus for detecting a vehicle contour based on point clouddata, which. includes: an acquiring' module, configured. toacquire to-be-trained point cloud data; a label data generatingmodule, configured to generate label data corresponding topoints in the to-be-trained point cloud data in response tolabeling on the points in the to-be-trained point cloud data,the labeling used to indicate whether each of the points inthe to-be-trained point cloud data is on a vehicle contour;a training module, configured to train a fully convolutionalneural network model based on the points in the to-be-trainedpoint cloud data and the label data corresponding to the pointsin the to-be-trained point cloud data, to obtain a vehicledetection model; and a detecting module, configured.to acquireto-be-detected point cloud data, and obtain a detection resultcorresponding to each to-be-detected point in theto-be-detected point cloud data based on the vehicle detection model.
    In some embodiments, the acquiring module is further configuredto: map to-be-trained three-dimensional data, collected by apoint cloud data collecting device, to generate a to-be-traineddepth. map, the values of respective pixel points in theto-be-trained depth map being respective points in theto-be-trained point cloud data corresponding to the to-be-trained three-dimensional data.
    In some embodiments, the to-be-trained three-dimensional dataD (x, y, z) is a coordinate in a coordinate system taking thepoint cloud data collecting device as an origin, an upwardvertical direction as a z axis, and any two axes perpendicularto each other in a horizontal plane as an x axis and a y axis;the coordinate of each pixel point in the to-be-trained depthmap is (r, c), and the value of each of the pixel points is (d, z); wherein 7/42 lO ßw/Aejßlø/Aølci= x2+y2 6 = atan2(y,x) d) = arcsin(z/W/x2 + yz + 22); G is an angle between a line connecting the to-be-trained three-dimensional data D to the origin and a first plane, A6is a horizontal resolution of the point cloud data collecting device, and the first plane is a plane where the y axis andthe z axis locate; and $ is an angle between a line connectingthe to-be-trained three-dimensional data D to the origin anda second.plane, Ao is a vertical resolution of the point clouddata collecting device, and the second plane is a plane where the x axis and the y axis locate.
    Ir1sonæaembodiments, the label data generating1nodule is furtherconfigured to: transform a point in the to-be-trained pointcloud data to generate two-dimensional label data with 26channels if the point is on the vehicle contour, wherein thevalue of a first channel in the 26 channels is l, the valueof a second channel in the 26 channels is O, and the 3“í- 24Üchannels in the 26 channels are respective rotation vectorsof coordinates of vertexes of a three-dimensional cube wherea vehicle is located with respect to a three-dimensionalcoordinate of the point on the vehicle contour in theto-be-trained point cloud data; and transform a point in theto-be-trained point cloud data to generate two-dimensionallabel data with 2 channels if the point is not on the vehiclecontour, wherein the value of a first channel in the 2 channelsis O, and the value of a second channel in the 2 channels is l.
    In some embodiments, the rotation vector (x“p, y'p, z'p) of acoordinate of a vertexe of a three-dimensional cube where a vehicle is located with respect to a three-dimensional 8/42 lO coordinate of the point on the vehicle contour in the to-be-trained point cloud data is: xp xp_xzv i T i y, R y, y,ZP Zp-Zl wherein R is a rotation matrix of a point P(xfl jq, zi) on the vehicle contour in the point cloud data, and:R I Rzzæzfleyi . 61. = atan2(yl. ,xl.) - 2@=mæm@/¶@-Uf+4Ü_ cos6, -sü16, 0Rá(6¿)= sü16, cos6, 00 0 1 cosóí 0 sü1QRyí= 0 1 0 -sü1$, 0 cosóí In some embodiments, the training module is further configuredto: initialize parameters of the fully convolutional neuralnetwork; adjust the parameters of the fully convolutionalneural network based on a loss function, wherein the lossfunction is a deviation between the label data correspondingto a current output and the label data corresponding to theto-be-trained point cloud data, and the current output is anoutput of the fully convolutional neural network correspondingto current to-be-trained point cloud data and currentparameters of the fully convolutional neural network; outputparameters corresponding to the loss function as parameters of the vehicle detection model if the loss function is at a 9/42 lO minimum value; and otherwise, return to execute the step ofadjusting the parameters of the fully convolutional neural network based on a loss function.
    In some embodiments, the detecting module is further configuredto: map to-be-detected three-dimensional data, which iscollectedkqftheepoint cloudwdata.collecting'device, to generatea to-be-detected depth map, the values of pixel points in theto-be-detected. depth. map corresponding' to respectiveto-be-detected points in the to-be-detected point cloud datacorresponding to the to-be-detected three-dimensional data;acquire a predicted.result of each of the to-be-detected pointsin the to-be-detected point cloud data based on the vehicledetection model, wherein the predicted result comprises aprobability that the to-be-detected point is on the vehiclecontour; and determine that a to-be-detected point is on thevehicle contour if the probability that the to-be-detectedpoint is on the vehicle contour is greater than a preset threshold.
    In some embodiments, the predicted result further compriseslocation information of the to-be-detectecípoint; the detectingmodule is further configured to: acquire first to-be-detectedpoints whose probabilities are each greater than the presetthreshold in the to-be-detected point cloud data, to form aset of the first to-be-detected points; and generate spatialvehicle location information corresponding to theto-be-detected point cloud data based on the locationinformation of the to-be-detecteclpoints in the set of the first to-be-detected points.
    The method and apparatus for detecting a vehicle contour basedon point cloud data according to the present application trainthe fully convolutional neural network1nodel by using the pointcloud data collected.by the point cloud data collecting deviceand the labeling for the point cloud data, to obtain the vehicle detection model suitable for the vehicle contour detection, 10/42 lO and use the vehicle detection model to determine whether thepoint cloud data contains a vehicle contour, thereby overcomingthe difficulty in detection and the detection error caused bythe two-dimensional size change of the vehicle during thevehicle contour is detected by using traditional two-dimensional images.
    In addition, in some embodiments, the solutions of the presentapplication can directly obtain the spatial locationinformation automatic driving of a driven vehicle.
    BRIEF DESCRIPTION OF THE DRAWINGS After reading detailed descriptions of non-limitingembodiments with reference to the following accompanyingdrawings, other features, objectives and advantages of the present application will be more apparent: FIG. l is a diagram showing an exemplary system architecture in which the present application may be applied; FIG. 2 is a flow chart of the method for detecting a vehiclecontour based on point cloud data according to an embodiment of the present application; FIG. 3 is a schematic flow chart of an alternativeimplementatior1of trainingeafully convolutional neural networkmodel based on the points in the to-be-trained point cloud dataand the label data corresponding to the points in theto-be-trained point cloud data to obtain a vehicle detection model in FIG. 2; FIG. 4 is a schematic flow chart of an alternativeimplementation of acquiring to-be-detected point cloud data,and obtaining a detection result corresponding to eachto-be-detected point in the to-be-detected point cloud data based on the vehicle detection model in FIG. 2; ll / 42 FIG. 5 is a schematic structural view of an apparatus fordetectingzavehicle contour based to an embodiment of the present application; and FIG. 6 is a schematic structural view of a computer systemadapted to implement a terminal apparatus or a server of the embodiments of the present application.
    DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS The present application will be further described below indetail in combination with the accompanying drawings and theembodiments. It should. be appreciated. that the specificembodiments described herein are merely used for explainingthe relevant invention, rather than limiting the invention.In addition, it should. be noted that, for the ease ofdescription, only the parts related to the relevant invention are shown in the accompanying drawings.
    It should also be noted that the embodiments in the presentapplication.and.the features in the embodiments may be combinedwitheachothercn1anon-conflictbasis.Thepresentapplicationwill be described. below in detail with. reference to the accompanying drawings and in combination with the embodiments.
    Fig. 1 shows an exemplary architecture of a system 100 whichmay be used by a method for detecting a vehicle contour basedon point cloud data or an apparatus for detecting a vehiclecontour based on point cloud data according to an embodiment of the present application.
    As shown in FIG. 1, the system architecture 100 may includeterminal devices 101, 102 and 103, a network 104 and a server105. The network 104 serves as a medium providing acommunication link between the terminal devices 101, 102 and103 and the server 105. The network 104 may include varioustypes of connections, such as wired or wireless communication links, or optical fibers. 12 / 42 The user may use the terminal devices 101, 102 and 103 tointeract with the server 105 through the network 104, in order to transmit or receive messages, etc.
    In some application scenarios, the terminal devices 101, 102,and 103 may be any electronic devices capable of collectingpoint cloud data. For example, the terminal devices 101, 102, and 103 may be three-dimensional laser scanners.
    Alternatively, in some other application scenarios, theterminal devices 101, 102, and 103 may perform datacommunication with a point cloud data collecting device toacquire the point cloud data collected.by the point cloud datacollecting device. For example, the terminal devices 101, 102,and 103 may be various electronic devices including, but notlimited to, a smart phone, a tablet computer, an e-book reader,a Moving Picture Experts Group Audio Layer III (MP3) player,a Moving Picture Experts Group Audio Layer IV (MP4) player, a laptop portable computery a desktop computer, and on the like.
    The server 105 may be a server providing various services, forexample, a background server for processing and analyzing thepoint cloud data acquired by the terminal devices 101, 102,and 103. The background server may perform processing, suchas analysis, on the received point cloud data, and feed theprocessing result (for example, whether the point cloud data includes a vehicle contour) back to the terminal devices.
    It should be noted that the method for detecting a vehiclecontour based on point cloud data according to the embodimentsof the present application may be executed by the server 105,and may also be executed by the terminal devices 101, 102, or103. Accordingly, the apparatus for detectingeavehicle contourbased on point cloud data may be set in the server 105, and may also be set in the terminal devices 101, 102, or 103.
    It should be appreciated that the numbers of the terminal devices, the networks and the servers in Fig. 1 are merely 13/42 illustrative. Any number of terminal devices, networks and servers may be provided based on the actual requirements.
    Referring to FIG. 2, it shows a process 200 of a method fordetectingzavehicle contour based the following steps.
    Step 210, acquire to-be-trained point cloud data.
    In the embodiment, an electronic device (e.g., the server orterminal device as shown in Fig. 1) where the method fordetecting a vehicle contour based on point cloud data runs mayacquire the point cloud data fron1a or to be developed.
    The point cloud is a set of mass points for expressing thespatial distribution and surface characteristics of an objectobtained by acquiring a spatial coordinate of each samplingpoint picture.
    Step 220, generate label data corresponding to points in theto-be-trained point cloud data in response to labeling on thepoints in the to-be-trained point cloud data, wherein thelabeling is used to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour. 14/42 lO In some alternative implementations, points in theto-be-trained point cloud.data.ma§fbe labeleclbywneans of1nanualaids, to indicate whether each point in the to-be-trained point cloud data is on a vehicle contour.
    Since the labeling is used to indicate whether each point inthe to-be-trained point cloud data is on a vehicle contour,the label data generated. based. on the labeling' contains information on whether a point is on the vehicle contour.
    Step>230, trainzafullyfconvolutional neural network1nodel basedon the points in the to-be-trained point cloud data and thelabel data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model.
    Generally, the basic structure of the Convolutional NeuralNetworks (CNN) model includes two layers. One layer is a featureextracting layer, in which the input of each neuron isassociated with a local accepted domain in the previous layer,and the local feature is extracted. Once the local feature isextracted, the spatial relationship between the local featureand other features may be determined subsequently. The otherlayer is a feature mapping layer, in which each calculationlayer of the network is composed.of multiple feature mappings,each feature mapping is a plane, and all the neurons on the plane have the same weighting value.
    In some alternative implementations, the to-be-trained pointcloud data may be inputted into the CNN model, and parametersoftheCNNnmdelcanbeiierativelyadjustedbasedcn1adeviationbetween. the output of the CNN' model and. the label datacorresponding to the to-be-trained point cloud data, so as tominimize the deviation between the output of the trained CNNmodel and the label data corresponding to the to-be-trained point cloud data.
    Step 240, acquire to-be-detected point cloud data, and obtain a detection result corresponding to each to-be-detected point 15/42 in the to-be-detected point cloud data based on the vehicle detection model.
    Through steps 210-240, a vehicle detection. model can beobtainedd In step 240, each to-be-detected. point in theto-be-detectecípoint cloud»data1naykmæinputted.into the vehicledetection model, to detect whether the to-be-detected pointcloud data contains a vehicle contour. The method for detectinga vehicle contour based on point cloud data of the presentembodiment uses three-dimensional.point cloud data as the basisfor detection, thereby avoiding the difficulty in detectionand detection error caused by the change of the two-dimensionalsize of a vehicle during the vehicle contour is detected by using traditional two-dimensional images.
    In some alternative implementations, the acquiring of to-be-trained point cloud data in step 210 may includes: Step 211, map to-be-trained three-dimensional data, collectedby a point cloud data collecting device, to generate ato-be-trained.depth map, the values of respective pixel pointsin the to-be-trained depth map being respective points in theto-be-trained point cloud data corresponding to the to-be-trained three-dimensional data.
    In some application scenarios of these alternativeimplementations, the to-be-trained three-dimensional data D(x, y, z) may be a coordinate in a coordinate system takingthe point cloud data collecting device as an origin, an upwardvertical direction as a z axis, and any two axes perpendicularto each other in a horizontal plane as an x axis and a y axis.Moreover, in the to-be-trained.depth map obtained on the basisof the to-be-trained three-dimensional data D, the coordinateof each pixel point in the to-be-trained depth map is (r, c), and the value of each of the pixel points is (d, z),wherein: 16/42 WLQ/Aejßbfi/Aøld= x2+y2 6 = atan2( y, x) d) = arcsin(z//x2 +y2 +22) (l) That is, in the depth map, the value in the position at the rm row and the cm column is (d, Z).
    G is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a first plane, A6is a horizontal resolution of the point cloud data collectingdevice, and the first plane is a plane where the y axis andthe z axis locate. That is, 6 may be considered as a "driftangle" of the to-be-trained three-dimensional data D with respect to the vertical plane. $ is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a second plane, Aois a vertical resolution of the point cloud data collectingdevice, and the second plane is a plane where the x axis andthe y axis locate. That is, $ may be considered as a "pitch angle" of the to-be-trained three-dimensional data D with respect to the horizontal plane.
    In addition, the meaning of the atan2(y, x) is an angle betweena ray starting from the coordinate origin and pointing to thepoint (x, y) and the positive direction of the x axis in the coordinate plane where the x axis and the y axis locate.
    In some alternative implementations, the generating of labeldata corresponding to points in the to-be-trained point clouddata in response to labeling on the points in the to-be-trained point cloud data in step 220 may include: Step 221, transform a point in the to-be-trained point cloud 17/42 lO data to generate two-dimensional label data with 26 channelsif the point is on the vehicle contour, wherein the value ofa first channel in the 26 channels is l, the value of a secondchannel in the 26 channels is O, and the 3“í- 24Ü1channels inthe 26 channels are respective rotation vectors of coordinatesof vertexes of a three-dimensional cube where a vehicle islocated with respect to a three-dimensional coordinate of thepoint on the vehicle contour in the to-be-trained point cloud data.
    Step 222, transform a point in the to-be-trained point clouddata to generate two-dimensional label data with 2 channelsif the point is not on the vehicle contour, wherein the valueof a first channel in the 2 channels is O, and the value of a second channel in the 2 channels is l.
    That is, regardless of a certain point in the to-be-trainedpoint cloud data being on or not on the vehicle contour, inthe obtained two-dimensional label data, the value of the firstchannel may be used to represent the probability that the pointis on the vehicle contour, and the value of the second channelmay be used to represent the probability that the point is noton the vehicle contour. It can be seen that the values of thefirst and second channels are both non-negative numbers, and the sum of the values of the first and second channels is l.
    In some application scenarios of these alternativeimplementations, the to-be-trainedqooint cloudwdata1nay'includesome points that cannot be clearly determined to be on thevehicle contour or not. In these application scenarios,suitable values may be set at the first and second channelsin the two-dimensional label data corresponding to«each.of thesepoints, to reflect the probability that the point is on the vehicle contour.
    In some alternative implementations, the rotation vector (x'p,yflp, z'p) of a coordinate of a vertexe of a three-dimensional cube where a vehicle is located with respect to a 18/42 lO three-dimensional coordinate of the point on the vehiclecontour in the to-be-trained point cloud data may be expressed with the following formula (2).
    Specifically, (2) wherein R is a rotation matrix of a point P(xfl jq, zi) on the vehicle contour in the point cloud data, and:R I Rzzæzfleyi . 61. = atan2(yl. ,xl.) Q=am¶mj/¶xf+yf+zf)_ _cos6¿ -sü16¿ 0Rá(6¿)= sü16¿ cos6¿ 00 0 1 _cos$¿ 0 sü1QfRyl-(Öf) I 0 1 0 _-sü1$¿ 0 cos$¿_ Herein, Gi and óí may have the meanings similar to those of 6 and $ in the formula (l). That is, Gi is an angle between a line connecting any point P(xfl yq, zfl on the vehicle contour to the origin and the first plane, the first plane bing a planewhere the y axis and the z axis locate. óí is an angle between the line connecting any point P(xl~, yi, zi) on the vehicle contourto the origin and the second plane, the second plane being a plane where the x axis and the y axis locate. 19/42 l0 In some alternative implementations, step 230 in the methodfor detecting a vehicle contour based on point cloud data ofthe present embodiment may be implemented through a process 300 shown in FIG. 3.
    Step 310, initialize parameters of the fully convolutional neural network.
    Step 320, adjust the parameters of the fully convolutional neural network based on a loss function.
    Herein, the loss function may be a deviation between the labeldata corresponding to a current output and the label datacorresponding to the to-be-trained point cloud data, and thecurrent output is an output of the fully convolutional neuralnetworkcorrespondingixncurrentto-be-trainedpointclouddataand. current parameters of the fully' convolutional neural network.
    Step 330, determine whether the loss function corresponding to the current output is a minimum value.
    If yes, in step 340, output parameters corresponding to the loss function as parameters of the vehicle detection model.
    Otherwise, that is, if the loss function corresponding to thecurrent output is not a minimum value, the adjusting of theparameters of the fully convolutional neural network based ona loss function in step 320 is repeated.until the loss function is a minimum value.
    In some alternative implementations, the acquiringto-be-detected point cloud data, and obtaining a detectionresult corresponding to each to-be-detected point in theto-be-detected point cloud data based on the vehicle detectionmodel in step 240 may be implemented.througheaprocess 400 shown in FIG. 4.
    Step 4l0, map to-be-detected three-dimensional data, collected 20 / 42 lO by the point cloud data collecting device, to generate ato-be-detected depth map, the values of pixel points in theto-be-detected. depth. map corresponding' to respectiveto-be-detected points in the to-be-detected point cloud datacorresponding to the to-be-detected three-dimensional data.In some alternative implementations, the to-be-detected depthmap1nay<:orrespond to the to-be-detected three-dimensional datacollected by the point cloud data collecting device and maybe obtained through the above formula (l).
    Step 420, acquire a predicted result of each of theto-be-detected points in the to-be-detected point cloud databased on the vehicle detection model, wherein the predictedresult comprises a probability that the to-be-detected point is on the vehicle contour.
    Step 430, determine that a to-be-detected point is on thevehicle contour if the probability that the to-be-detectedpoint is on the vehicle contour is greater than a preset threshold.
    In some optional implementation manners, the predicted resultmay further include location information of the to-be-detected points.
    In these alternative implementations, the process 400 may further include: Step 440, acquire first to-be-detected points whoseprobabilities are each greater than the preset threshold inthe to-be-detected point cloud data, to fornla set of the firstto-be-detected points.
    Step 450, generate spatial vehicle location informationcorresponding to the to-be-detected point cloud data based onthe location information of the to-be-detected points in the set of the first to-be-detected points.
    In some application scenarios, the detection result obtained 2l/42 lO after the to-be-detected point cloud data is inputted into thevehicle detection model may have the same representation formas the two-dimensional label data of the points in theto-be-trained point cloud data, which are on the vehiclecontour. That is, the detection result also has 26 channels,in which the value of the first channel represents theprobability that the to-be-detected point is on the vehiclecontour based on the current vehicle detection model, and thevalue of the second channel represents the probability thatthe to-be-detected point is not on the vehicle contour basedon the current vehicle detection model. The values of the Bm 26” channels may be used to represent a spatial location of the to-be-detected point.
    In this way, the to-be-detected points each having a value ofthe first channel greater than a preset threshold in thedetection result of the to-be-detected point cloud data areconsidered as the points on the vehicle contour. Through atechnique such as data fusion, the location where the vehiclelocatesi11theto-be-detectedpointclouddatarmnfbereproducedaccording to the spatial locations of the points on the vehicle contour.
    The method according to the embodiments of the presentapplication trains the fully convolutional neural network1nodelby using the point cloud data collected.by the point cloud datacollecting device and the labeling for the point cloud data,to obtain the vehicle detection model suitable for the vehiclecontour detection, and uses the vehicle detection model todetermine whether the point cloud data contains a vehiclecontour, thereby overcoming the difficulty in detection andthe detection error caused by the two-dimensional size changeof the vehicle during the vehicle contour is detected.by using traditional two-dimensional images.
    Further referring to FIG. 5, as an implementation for the methods shown in the above figures, an embodiment of the present 22 / 42 application provides an apparatus for detecting a vehiclecontour based on point cloud data, the apparatus in thisembodiment corresponds to the method of the embodiment shownin FIG. 2, and the apparatus may be specifically applied in a variety of electronic devices.
    As shown in FIG. 5, the apparatus 500 in the present embodimentincludes an.acquiring1nodule 510, a label data.generating1nodule 520, a training module 530, and a detecting module 540.
    Specifically, the acquiring module 510 may be configured to acquire to-be-trained point cloud data.
    The label data generating module 520 may be configured togenerate label data corresponding to points in theto-be-trained point cloud data in response to labeling on thepoints in the to-be-trained point cloud data, the labeling usedto indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour.
    The training module 530 may be configured to train a fullyconvolutional neural network model based on the points in theto-be-trained point cloud data and the label data correspondingto the points in the to-be-trained point cloud data, to obtain a vehicle detection model.
    The detecting module 540 may be configured to acquireto-be-detected point cloud data, and obtain a detection resultcorresponding to each to-be-detected point in theto-be-detected point cloud data based on the vehicle detection model.
    In some alternative implementations, the acquiring module 510may be further configured to: map to-be-trainedthree-dimensional data, collected. by a point cloud datacollecting device, to generate a to-be-trained depth map, thevalues of respective pixel points in the to-be-trained depth map being respective points in the to-be-trained point cloud 23/42 lO data corresponding' to the to-be-trained. three-dimensional data.
    In some alternative implementations, the to-be-trainedthree-dimensional data Z) (x, 34 z) is a coordinate in acoordinate systenltaking the point cloud data collecting deviceas an origin, an upward vertical direction as a z axis, andany two axes perpendicular to each other in a horizontal planeas an x axis and a y axis; the coordinate of each pixel pointin the to-be-trained.depth map is (r, c), and the value of each of the pixel points is (d, z); wherein: ßw/Aejßlø/Aølci= x2+y2 6 = atan2(y,x) d) = arcsin(z/W/x2 + yz + 22); G is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a first plane, A6is a horizontal resolution of the point cloud data collectingdevice, and the first plane is a plane where the y axis andthe z axis locate; and $ is an angle between a line connectingthe to-be-trained three-dimensional data D to the origin anda second.plane, Ao is a vertical resolution of the point clouddata collecting device, and the second plane is a plane where the x axis and the y axis locate.
    In some alternative implementations, the label data generatingmodule 520 may be further configured to: transform.a point inthe to-be-trained point cloud data to generate two-dimensionallabel data with 26 channels if the point is on the vehiclecontour, wherein.the value of a first channel in the 26 channelsis l, the value of a second channel in the 26 channels is O,and the 3mc- 24” channels in the 26 channels are respective rotation vectors of coordinates of vertexes of a 24/42 lO three-dimensional cube where a vehicle is located with respectto a three-dimensional coordinate of the point on the vehiclecontour in the to-be-trained point cloud data; and transforma point in the to-be-trained point cloud data to generatetwo-dimensional label data with 2 channels if the point is noton the vehicle contour, wherein the value of a first channelin the 2 channels is O, and the value of a second channel in the 2 channels is l.
    In some alternative implementations, the rotation vector (x'p,yflp, z'p) of a coordinate of a vertexe of a three-dimensionalcube where a vehicle is located with respect to athree-dimensional coordinate of the point on the vehicle contour in the to-be-trained point cloud data is: wherein R is a rotation matrix of a point P(xfl jq, zi) on the vehicle contour in the point cloud data, and:R I Rzz (GÛRyz 61. = atan2(yl. ,xl.) @=am¶mj/¶@2+%2+4Ü I _cos6¿ -sü16¿ 0-Rá(6¿)= sü16¿ cos6¿ 00 0 1 _cos$¿ 0 sü1QfRyl-(Öf) I 0 1 0 _-sü1$¿ 0 cos$¿_ In some alternative implementations, the training module 530 25/42 lO maykmæfurther configured to: initialize parameters of the fullyconvolutional neural network; adjust the parameters of thefully convolutional neural network based on a loss function,wherein the loss function is a deviation between the label datacorresponding to a current output and the label datacorresponding to the to-be-trained point cloud data, and thecurrent output is an output of the fully convolutional neuralnetworkcorrespondingixncurrentto-be-trainedpointclouddataand. current parameters of the fully' convolutional neuralnetwork; output parameters corresponding to the loss functionas parameters of the vehicle detection model if the lossfunction is at a1ninimun1value; and otherwise, return to executethe step of adjusting the parameters of the fully convolutional neural network based on a loss function.
    In some alternative implementations, the detecting module 540may be further configured to: map to-be-detectedthree-dimensional data, which is collected by the point clouddata collecting device, to generate a to-be-detected depth.map,the values of pixel points in the to-be-detected depth mapcorresponding' to respective to-be-detected. points in theto-be-detected point cloud data corresponding to theto-be-detected three-dimensional data; and acquire a predictedresult of each of the to-be-detected points in theto-be-detected point cloud data based on the vehicle detection model.
    Herein, the predicted result may include a probability thatthe to-be-detected point is on the vehicle contour. If theprobability that the to-be-detected point is on the vehiclecontour is greater than a preset threshold, it is determined that a to-be-detected point is on the vehicle contour.
    In some alternative implementations, the predicted result mayfurther include location information of the to-be-detected point.
    In these alternative implementations, the detecting module 540 26/42 may be further configured to: acquire first to-be-detectedpoints whose probabilities are each greater than the presetthreshold in the to-be-detected point cloud data, to form aset of the first to-be-detected points; and generate spatialvehicle location information corresponding to theto-be-detected point cloud data based on the locationinformation of the to-be-detecteclpoints in the set of the first to-be-detected points.
    Those skilled in the art may understand that the apparatus 500for detecting a vehicle contour based on point cloud datafurther includes some commonly known structures, such as aprocessor, and a næmory. In order to avoid unnecessarilyobscuring the embodiments disclosed in this disclosure, these commonly known structures are not shown in FIG. 5.
    Referring to Fig. 6, a schematic structural diagram of acomputer system 600 adapted to implement a terminal apparatusor a server of the embodiments of the present application is shown.
    As shown in Fig. 6, the computer system 600 includes a centralprocessing unit (CPU) 601, which may execute variousappropriate actions and processes in accordance with a programstored.in a read-only memory (ROM) 602 or a program.loaded intoa random access memory (RAM) 603 from a storage portion 608.The RAM 603 also stores various programs and data required byoperations of the system 600. The CPU 601, the ROM 602 and theRAM 603 are connected to each other through a bus 604. Aninput/output (I/O) interface 605 is also connected to the bus604.
    The following components are connected to the I/O interface605: an input portion 606 including a keyboard, a mouse etc.;an output portion 607 comprising a cathode ray tube (CRT), aliquid crystal display device (LCD), a speaker etc.; a storageportion 608 including' a hard disk and the like; and a communication portion 609 comprisingzanetwork interface card, 27 / 42 such as a LAN card and a modem. The communication portion 609performs communication processes via a network, such as theInternet. A driver 610 is also connected to the I/O interface605 as required. A removable medium 611, such as a magneticdisk, an optical disk, a magneto-optical disk, and asemiconductor memory, may be installed on the driver 610, tofacilitate the retrieval of a computer* program. from. theremovable medium 611, and the installation thereof on the storage portion 608 as needed.
    In particular, according to an embodiment of the presentdisclosure, the process described.above with reference to Fig.2 may be implemented in a computer software program. Forexample, an embodiment of the present disclosure includes acomputer program product, which comprises a computer programthat is tangibly embedded in a machine-readable medium. Thecomputer program comprises program codes for executing themethod of Fig. 2. In such an embodiment, the computer programmay' be downloaded and installed from. a network via thecommunication portion 609, and/or may be installed from the removable media 611.
    The flowcharts and block diagrams in the figures illustratearchitectures, functionsznuíoperations that1naykmæimplementedaccording to the system, the method and the computer programproduct of the various embodiments of the present invention.In this regard, each.block in the flow charts and.block diagramsmay represent a module, a program segment, or a code portion.The module, the program segment, or the code portion comprisesone or more executable instructions for implementing' thespecified logical function. It should be noted that, in somealternative implementations, the functions denoted by theblocks may occur in a sequence different from the sequencesshown in the figures. For example, in practice, two blocks insuccession may be executed, depending on the involvedfunctionalities, substantially in parallel, or in a reverse sequence. It should.also be noted.that, each.block in the block 28/42 lO diagrams and/or the flow charts and/or a combination of theblocks may be implemented by a dedicated hardware-based systemexecuting specific functions or operations, or by'a combination of a dedicated hardware and computer instructions.
    The units or modules involved in the embodiments of the presentapplication may be implemented by way of software or hardware.The described units or modules may also be provided in aprocessor, for example, described as: a processor, comprisingan acquiring module, a label data generating module, a trainingmodule and a detecting module, where the names of these unitsor modules are not considered as a limitation to the units ormodules. For example, the acquiring unit may also be described as "a unit for acquiring to-be-trained point cloud data".
    In another aspect, the present application further providesa computer readable storage medium. The computer readablestorage medium may be the computer readable storage mediumincluded in the apparatus in the above embodiments, or astand-alone computer readable storage mediun1which.has not beenassembled into the apparatus. The computer readable storagemediun1stores one or1nore]orograms. When the one or1nore programsare executed by a device, the device is to: acquireto-be-trained point cloud data; generate label datacorresponding to points in the to-be-trained point cloud datain response to labeling on the points in the to-be-trainecípointcloud data, the labeling used to indicate whether each of thepoints in the to-be-trained point cloud data is on a vehiclecontour; traineafully convolutional neural network1nodel basedon the points in the to-be-trained point cloud data and thelabel data corresponding to the points in the to-be-trainedpoint cloud data, to obtain a vehicle detection model; and adetecting module, configured to acquire to-be-detected pointcloud data, and obtain a detection result corresponding to eachto-be-detected point in the to-be-detected point cloud data based on the vehicle detection model. 29/42 lO Theforegoingir;onlyzadescriptionof1ju2preferredembodimentsof the present application and the applied technicalprinciples. It should be appreciated by those skilled in theart that the inventive scope of the present application is notlimited to the technical solutions formed by the particularcombinations of the above technical features. The inventivescope should also cover other technical solutions formed byany combinations of the above technical features or equivalentfeatures thereof without departing from the concept of theinvention, such as, technical solutions formed by replacingthe features as disclosed.in the present application with (but not limited to), technical features with similar functions. 30 / 42
    权利要求:
    Claims (1)
    [1] 1. k) OO10"U'|»J> 1112 131416 k) U'|»J>(pl) OO10"U"|»J>UU WHAT IS CLAIMED IS: A method for detecting a vehicle contour based on point cloud data, comprising: acquiring to-be-trained point cloud data; generating label data corresponding to points in theto-be-trained point cloud data in response to labeling onthe points in the to-be-trained point cloud, the labelingused to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour; training a fully convolutional neural network model basedon the points in the to-be-trained point cloud data and thelabel data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and acquiring to-be-detected point cloud data, and obtaininga detection result correspondingixneach.to-be-detecteclpointin the to-be-detected point cloud data based on the vehicle detection model. The method according to claim 1, wherein the acquiring of to-be-trained point cloud data comprises: mapping to-be-trained three-dimensional data, collectedby a point cloud data collecting device, to generate ato-be-trained depth map, the values of respective pixelpoints in the to-be-trained depth.map being respective pointsin the to-be-trained point cloud data corresponding to the to-be-trained three-dimensional data. The method according to claim 2, wherein the to-be-trained three-dimensional data D (X, y, z) isa coordinate in a coordinate system taking the point clouddata collecting device as an origin, an upward vertical direction as a z axis, and any two axes perpendicular to each 31 / 42 11 12131416 17 18 19 21 CDLOOOQOWLTI »BUUNIÄ other in a horizontal plane as an x axis and a y axis; the coordinate of each pixel point in the to-be-traineddepth map is (r, c), and the value of each of the pixel points is (d, z) ; wherein ßw/Aejcaø/Aølci= x2+y2 6 = atan2(y,x) d) = arcsin(z/W/x2 + yz + 22) _ 6 is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a first plane,A6 is a horizontal resolution of the point cloud datacollecting device, and the first plane is a plane where the y axis and the z axis locate; and $ is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a second plane,Ao is a vertical resolution of the point cloud datacollecting device, and the second.plane is a plane where the x axis and the y axis locate. The1nethod.according to clain13, wherein the generating of labeldata corresponding to points in the to-be-trained point clouddata in response to labeling on the points in the to-be-trained point cloud data comprises: transformingeapointi11the to-be-traiïmxípoint cloud datato generate two-dimensional label data with 26 channels ifthe point is on the vehicle contour, wherein the value ofa first channel in the 26 channels is 1, the value of a secondchannel in the 26 channels is O, and the 3“í- 24” channels in the 26 channels are respective rotation vectors of 32/42 11121314 16171819 OïUïßbuN) 11 12 coordinates of vertexes of a three-dimensional cube wherea vehicle is located with respect to a three-dimensionalcoordinate of the point on the vehicle contour* in the to-be-trained point cloud data; and transformingeapointi11the to-be-traiïmxípoint cloud datato generate two-dimensional label data with 2 channels ifthe point is not on the vehicle contour, wherein the valueof a first channel in the 2 channels is O, and the value of a second channel in the 2 channels is 1. The method according to claim 3, wherein the rotation vector (x“p, y'p, z'p) of a coordinate of avertexe ofeithree-dimensional cube whereeavehicle is locatedwith respect to a three-dimensional coordinate of the pointon the vehicle contour in the to-be-trained point cloud data is: wherein R is a rotation matrix of a point P(xfl ;q, zg on the vehicle contour in the point cloud data, and:R I Rzzæzflayi _ 61. = atan2(yl. ,xl.) - 2@=mæm@/¶@-Uf+4Ü_ cos6¿ -sü16¿ 0Rá(6¿)= sü16¿ cos6¿ 00 0 1 33/42 13 Oï U'|»J>UUl)|-* 11121314 161718 19 Oï U'|»J>UUl)|-* cosóí 0 sü1QR,,<<1>,->= 0 1 0 -sü1$, 0 cosóí The method according to clain11, wherein.the training of a fullyconvolutional neural network model based on the points in theto-be-trained point cloud data and the label data correspondingto the points in the to-be-trained.point cloud data, to obtain a vehicle detection model comprises: initializing parameters of the fully convolutional neural network; adjusting the parameters of the fully convolutional neuralnetwork based on a loss function, wherein the loss functionis a deviation between the label data corresponding to acurrent output and the label data corresponding to theto-be-trained point cloud data, and the current output isan output of the fully convolutional neural networkcorresponding to current to-be-trained point cloud data and current parameters of the fully convolutional neural network; outputting parameters corresponding to the loss functionas parameters of the vehicle detection model if the loss function is at a minimum value; and otherwise, returning to execute the step of adjusting theparameters of the fully convolutional neural network based on a loss function. The method according to claim 5, wherein the acquiring ofto-be-detected point cloud data, and obtaining of a detectionresult corresponding to each to-be-detected point in theto-be-detected point cloud data based on the vehicle detection model comprises: mapping to-be-detected three-dimensional data, which is collected by the point cloud data collecting device, to 34 / 42 1112 13141617 1819 10"U'|»J> 11 121314 generate a to-be-detected depth map, the values of pixelpoints in the to-be-detected depth map corresponding torespective to-be-detected points in the to-be-detected pointcloud data corresponding to the to-be-detected three-dimensional data; acquiringe1predicted.result of each of the to-be-detectedpoints in the to-be-detected point cloud data based on thevehicle detection model, wherein the predicted resultcomprises a probability that the to-be-detected point is on the vehicle contour; and determining that a to-be-detected point is on the vehiclecontour if the probability that the to-be-detected point is on the vehicle contour is greater than a preset threshold. The method according to claim 7, wherein the predicted. result further comprises location information of the to-be-detected point; the acquiring of to-be-detected point cloud data, andobtaining' of a detection result corresponding' to eachto-be-detected point in the to-be-detected point cloud data based on the vehicle detection model further comprises: acquiringfirstto-be-detectedpointswhoseprobabilitiesare each greater than the preset threshold in theto-be-detected point cloud data, to form a set of the firstto-be-detected points; and generating spatial vehicle location informationcorresponding to the to-be-detected point cloud data basedon the location information of the to-be-detected points in the set of the first to-be-detected points. An apparatus for detecting a vehicle contour based on point cloud data, comprising: at least one processor; and 35 / 42 1112 131416 171819 k) OO10"U'|»J>UU 11121314 a memory storing instructions, which when executed.by theat least one processor, cause the at least one processor to perform operations, the operations comprising: acquiring to-be-trained point cloud data; generating label data corresponding to points in theto-be-trained point cloud data in response to labeling onthe points in the to-be-trainedqooint cloudwdata, the labelingused to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour; training a fully convolutional neural network model basedon the points in the to-be-trained point cloud data and thelabel data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and acquiring to-be-detected point cloud data, and obtaininga detection result correspondingixneach.to-be-detecteclpointin the to-be-detected point cloud data based on the vehicle detection model. The apparatus according to claim 9, wherein the acquiring of to-be-trained point cloud data comprises: mapping to-be-trained three-dimensional data, collectedby a point cloud data collecting device, to generate ato-be-trained depth map, the values of respective pixelpoints in the to-be-trained depth.map being respective pointsin the to-be-trained point cloud data corresponding to the to-be-trained three-dimensional data. The apparatus according to claim 10, wherein the to-be-trained three-dimensional data D (X, y, z) isa coordinate in a coordinate system taking the point clouddata collecting device as an origin, an upward verticaldirection as a z axis, and any two axes perpendicular to each other in a horizontal plane as an x axis and a y axis; 36 / 42 1617 18 19 21 222324 26 27 28 29 OO10"U'| »BUUNIA 1112 12. the coordinate of each pixel point in the to-be-traineddepth map is (r, c), and the value of each of the pixel points is (d, z) ; wherein,ßw/Aejcaø/Aølci= x2+y2 6 = atan2(y,x) d) = arcsin(z/W/x2 + yz + 22) _ 6 is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a first plane,A6 is a horizontal resolution of the point cloud datacollecting device, and the first plane is a plane where the y axis and the z axis locate; and $ is an angle between a line connecting the to-be-trainedthree-dimensional data D to the origin and a second plane, Ao is a vertical resolution of the point cloud data collecting device, and the second.plane is a plane where the x axis and the y axis locate. The apparatus according to claim 11, wherein the generatingoflabeldatacorresponding1x>pointsiJ1theto-be-trainedpointin the cloud data in response to labeling on the points to-be-trained point cloud data comprises: transformingeapointi11the to-be-traiïmxípoint cloud datato generate two-dimensional label data with 26 channels ifthe point is on the vehicle contour, wherein the value ofa first channel in the 26 channels is 1, the value of a secondchannel in the 26 channels is O, and the 3“í- 24” channelsin the 26 channels are respective rotation vectors ofcoordinates of vertexes of a three-dimensional cube where a vehicle is located with respect to a three-dimensional 37/42 1314 16171819 OïUïßbuN) 11 12 1314 coordinate of the point on the vehicle contour* in the to-be-trained point cloud data; and transformingeapointi11the to-be-traiïmxípoint cloud datato generate two-dimensional label data with 2 channels ifthe point is not on the vehicle contour, wherein the valueof a first channel in the 2 channels is O, and the value of a second channel in the 2 channels is 1. The apparatus according to claim 11, wherein the rotation vector (x“p, y'p, z'p) of a coordinate of avertexe ofeithree-dimensional cube whereeavehicle is locatedwith respect to a three-dimensional coordinate of the pointon the vehicle contour in the to-be-trained point cloud data is: Xp xp-xl' _ T yp -R yp-yl ZP Zp_ZZ wherein R is a rotation matrix of a point P(xfl ;q, zg on the vehicle contour in the point cloud data, and:R I Rzzæzflayi , 61. = atan2(yl. ,xl.) @=am¶mj/¶xf+yf+zf)_ cos6¿ -sü16¿ 0Rá(6¿)= sü16¿ cos6¿ 00 0 1 Error! Objects cannot be created from editing field codes.. 38/42 Oï U'|»J>UUl)|-* 11121314 161718 19 |-*|-*I-*CDLOOOQOÄ LT|J>UUl)|-* 14. 15. The apparatus according to claim 9, wherein the training ofa fully convolutional neural network model based on the pointsin the to-be-trained point cloud data and the label datacorresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model comprises: initializing parameters of the fully convolutional neural network; adjusting the parameters of the fully convolutional neuralnetwork based on a loss function, wherein the loss functionis a deviation between the label data corresponding to acurrent output and the label data corresponding to theto-be-trained point cloud data, fully and the current output is an output of the convolutional neural networkcorresponding to current to-be-trained point cloud data and current parameters of the fully convolutional neural network; outputting parameters corresponding to the loss functionas parameters of the vehicle detection model if the loss function is at a minimum value; and otherwise, returning to execute the step of adjusting theparameters of the fully convolutional neural network based on a loss function. The apparatus according to claim 13, wherein the acquiring ofto-be-detected point cloud data, and obtaining of a detectionresult corresponding to each to-be-detected point in theto-be-detected point cloud data based on the vehicle detection model comprises: mapping to-be-detected three-dimensional data, which iscollected by the point cloud data collecting device, togenerate a to-be-detected depth map, the values of pixelpoints in the to-be-detected depth map corresponding torespective to-be-detected points in the to-be-detected pointdata to-be-detected cloud corresponding to the 39/42 12 13141617 1819 10"U'|»J> 11 121314 16171819 21 16. 17. three-dimensional data; acquiringe1predicted.result of each of the to-be-detectedpoints in the to-be-detected point cloud data based on thevehicle detection model, wherein the predicted resultcomprises a probability that the to-be-detected point is on the vehicle contour; and determining that a to-be-detected point is on the vehiclecontour if the probability that the to-be-detected point is on the vehicle contour is greater than a preset threshold. The apparatus according to claim 15, wherein the predicted. result further comprises location information of the to-be-detected point; the acquiring of to-be-detected point cloud data, andobtaining' of a detection result corresponding' to eachto-be-detected point in the to-be-detected point cloud data based on the vehicle detection model further comprises: acquiringfirstto-be-detectedpointswhoseprobabilities are each greater than the preset threshold in theto-be-detected point cloud data, to form a set of the first to-be-detected points; and generating spatial vehicle location information corresponding to the to-be-detected point cloud data basedon the location information of the to-be-detected points in the set of the first to-be-detected points. A non-transitory storage medium storing one or more programs,the one or more programs when executed by an apparatus, causingthe apparatus to perform a næthod for detecting a vehicle contour based on point cloud data, comprising: acquiring to-be-trained point cloud data; generating label data corresponding to points in the 40/42 222324 26272829 3l3233 to-be-trained point cloud data in response to labeling onthe points in the to-be-trained point cloud, the labelingused to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour; training a fully convolutional neural network model basedon the points in the to-be-trained point cloud data and thelabel data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and acquiring to-be-detected point cloud data, and obtaininga detection result correspondingixneach.to-be-detecteclpointin the to-be-detected point cloud data based on the vehicle detection model. 4l/42
    类似技术:
    公开号 | 公开日 | 专利标题
    SE1651131A1|2017-07-28|Method and apparatus for detecting vehicle contour based on point cloud data
    US11163991B2|2021-11-02|Method and apparatus for detecting body
    CN109117831B|2021-10-12|Training method and device of object detection network
    WO2018205467A1|2018-11-15|Automobile damage part recognition method, system and electronic device and storage medium
    JP2018063236A|2018-04-19|Method and apparatus for annotating point cloud data
    US20180240194A1|2018-08-23|Visual analytics based vehicle insurance anti-fraud detection
    US10832478B2|2020-11-10|Method and system for virtual sensor data generation with depth ground truth annotation
    US10614592B2|2020-04-07|Three-dimensional posture estimating method and apparatus, device and computer storage medium
    US10846870B2|2020-11-24|Joint training technique for depth map generation
    CN109344804A|2019-02-15|A kind of recognition methods of laser point cloud data, device, equipment and medium
    US10242463B2|2019-03-26|Providing visualization data to a co-located plurality of mobile devices
    US10762681B2|2020-09-01|Map generation system and method for generating an accurate building shadow
    KR20200045701A|2020-05-06|Learning method of detecting vanishing point, method and apparatus of detecting vanishing point
    CN109116374B|2021-08-17|Method, device and equipment for determining distance of obstacle and storage medium
    CN110059623B|2021-06-11|Method and apparatus for generating information
    CN112580457A|2021-03-30|Vehicle video processing method and device, computer equipment and storage medium
    Ichihashi et al.2019|Estimating visibility of annotations for view management in spatial augmented reality based on machine-learning techniques
    CN111311485A|2020-06-19|Image processing method and related device
    CN109118532B|2020-11-20|Visual field depth estimation method, device, equipment and storage medium
    US20210312195A1|2021-10-07|Lane marking detecting method, apparatus, electronic device, storage medium, and vehicle
    CN110770540A|2020-02-07|Method and device for constructing environment model
    CN109815307B|2021-07-09|Position determination method, apparatus, device, and medium
    Jia et al.2020|A new disparity map quality assessment based on structural similarity for remotely sensed image pairs
    CN114049396A|2022-02-15|Method and device for marking training image and tracking target, electronic equipment and medium
    CN112396059A|2021-02-23|Certificate identification method and device, computer equipment and storage medium
    同族专利:
    公开号 | 公开日
    KR101836304B1|2018-03-08|
    KR20170089743A|2017-08-04|
    IL247446D0|2016-12-29|
    DE102016116818A1|2017-07-27|
    SE541962C2|2020-01-14|
    IL247446A|2020-11-30|
    US20170213093A1|2017-07-27|
    CN107025642B|2018-06-22|
    JP2017134814A|2017-08-03|
    CN107025642A|2017-08-08|
    US10229330B2|2019-03-12|
    JP6364049B2|2018-07-25|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    JP3238178B2|1990-12-27|2001-12-10|株式会社東芝|Learning machine learning method|
    US7586489B2|2005-08-01|2009-09-08|Nvidia Corporation|Method of generating surface defined by boundary of three-dimensional point cloud|
    US8332134B2|2008-04-24|2012-12-11|GM Global Technology Operations LLC|Three-dimensional LIDAR-based clear path detection|
    JP4864043B2|2008-05-13|2012-01-25|株式会社豊田中央研究所|Image processing apparatus, method, and program|
    US8411953B2|2008-09-30|2013-04-02|International Business Machines Corporation|Tagging images by determining a set of similar pre-tagged images and extracting prominent tags from that set|
    JP2011065400A|2009-09-17|2011-03-31|Daihatsu Motor Co Ltd|Object recognition apparatus|
    US8605998B2|2011-05-06|2013-12-10|Toyota Motor Engineering & Manufacturing North America, Inc.|Real-time 3D point cloud obstacle discriminator apparatus and associated methodology for training a classifier via bootstrapping|
    JP2013205983A|2012-03-27|2013-10-07|Sony Corp|Information input apparatus, information input method, and computer program|
    US8996228B1|2012-09-05|2015-03-31|Google Inc.|Construction zone object detection using light detection and ranging|
    JP6021689B2|2013-02-26|2016-11-09|三菱重工メカトロシステムズ株式会社|Vehicle specification measurement processing apparatus, vehicle specification measurement method, and program|
    US9730643B2|2013-10-17|2017-08-15|Siemens Healthcare Gmbh|Method and system for anatomical object detection using marginal space deep neural networks|
    WO2015083199A1|2013-12-04|2015-06-11|J Tech Solutions, Inc.|Computer device and method executed by the computer device|
    CN103871079B|2014-03-18|2016-11-09|南京金智视讯技术有限公司|Wireless vehicle tracking based on machine learning and light stream|
    US9460517B2|2014-10-22|2016-10-04|Pointivo, Inc|Photogrammetric methods and devices related thereto|
    EP3213292A4|2014-10-28|2018-06-13|Hewlett-Packard Development Company, L.P.|Three dimensional object recognition|
    CN104463241A|2014-10-31|2015-03-25|北京理工大学|Vehicle type recognition method in intelligent transportation monitoring system|
    CN104537387A|2014-12-16|2015-04-22|广州中国科学院先进技术研究所|Method and system for classifying automobile types based on neural network|
    WO2016207875A1|2015-06-22|2016-12-29|Photomyne Ltd.|System and method for detecting objects in an image|
    US9710714B2|2015-08-03|2017-07-18|Nokia Technologies Oy|Fusion of RGB images and LiDAR data for lane classification|
    CA2995850A1|2015-08-31|2017-03-09|Cape Analytics, Inc.|Systems and methods for analyzing remote sensing imagery|
    CN105184271A|2015-09-18|2015-12-23|苏州派瑞雷尔智能科技有限公司|Automatic vehicle detection method based on deep learning|
    US10192129B2|2015-11-18|2019-01-29|Adobe Systems Incorporated|Utilizing interactive deep learning to select objects in digital visual media|
    EP3408848A4|2016-01-29|2019-08-28|Pointivo Inc.|Systems and methods for extracting information about objects from scene information|
    US9805274B2|2016-02-03|2017-10-31|Honda Motor Co., Ltd.|Partially occluded object detection using context and depth ordering|
    CN113238581A|2016-02-29|2021-08-10|星克跃尔株式会社|Method and system for flight control of unmanned aerial vehicle|
    US20170270406A1|2016-03-18|2017-09-21|Qualcomm Incorporated|Cloud-based processing using local device provided sensor data and labels|
    US20170312032A1|2016-04-27|2017-11-02|Arthrology Consulting, Llc|Method for augmenting a surgical field with virtual guidance content|
    DE202017102495U1|2016-05-02|2017-08-07|Google Inc.|Sharing vehicle settings data|
    US9886771B1|2016-05-20|2018-02-06|Ccc Information Services Inc.|Heat map of vehicle damage|
    US10055675B2|2016-06-15|2018-08-21|Ford Global Technologies, Llc|Training algorithm for collision avoidance using auditory data|
    US10311648B2|2016-06-22|2019-06-04|Aquifi, Inc.|Systems and methods for scanning three-dimensional objects|
    US10573018B2|2016-07-13|2020-02-25|Intel Corporation|Three dimensional scene reconstruction based on contextual analysis|
    EP3497550A4|2016-08-12|2020-03-11|Aquifi, Inc.|Systems and methods for automatically generating metadata for media documents|
    US10592805B2|2016-08-26|2020-03-17|Ford Global Technologies, Llc|Physics modeling for radar and ultrasonic sensors|CN107492098B|2017-08-17|2018-04-10|广东工业大学|It is a kind of based on PCA and CNN high-temperature forging surface defect in position detecting method|
    US10482628B2|2017-09-30|2019-11-19|United States Of America As Represented By The Secretary Of The Army|Photogrammetric point cloud compression for tactical networks|
    CN107610524A|2017-11-02|2018-01-19|济南浪潮高新科技投资发展有限公司|A kind of method and device that parking position Intelligent Recognition is carried out using laser radar|
    CN107944354B|2017-11-10|2021-09-17|南京航空航天大学|Vehicle detection method based on deep learning|
    CN108108657B|2017-11-16|2020-10-30|浙江工业大学|Method for correcting locality sensitive Hash vehicle retrieval based on multitask deep learning|
    CN108875902A|2017-12-04|2018-11-23|北京旷视科技有限公司|Neural network training method and device, vehicle detection estimation method and device, storage medium|
    US10705216B2|2017-12-07|2020-07-07|Institute For Information Industry|Three-dimensional point cloud tracking apparatus and method using recurrent neural network|
    CN108198268B|2017-12-19|2020-10-16|江苏极熵物联科技有限公司|Production equipment data calibration method|
    CN108225348B|2017-12-29|2021-08-24|百度在线网络技术（北京）有限公司|Map creation and moving entity positioning method and device|
    CN108415032B|2018-03-05|2021-06-25|中山大学|Point cloud semantic map construction method based on deep learning and laser radar|
    CN110634159A|2018-06-21|2019-12-31|北京京东尚科信息技术有限公司|Target detection method and device|
    CN108921105B|2018-07-06|2020-11-03|京东数字科技控股有限公司|Method and device for identifying target number and computer readable storage medium|
    CN109166128B|2018-07-31|2021-11-26|东华大学|Non-equivalent three-dimensional point cloud segmentation method|
    CN109101967A|2018-08-02|2018-12-28|苏州中德睿博智能科技有限公司|The recongnition of objects and localization method, terminal and storage medium of view-based access control model|
    CN109299656B|2018-08-13|2021-10-22|浙江零跑科技股份有限公司|Scene depth determination method for vehicle-mounted vision system|
    CN109188457B|2018-09-07|2021-06-11|百度在线网络技术（北京）有限公司|Object detection frame generation method, device, equipment, storage medium and vehicle|
    US11151734B2|2018-09-14|2021-10-19|Huawei Technologies Co., Ltd.|Method and system for generating synthetic point cloud data using a generative model|
    CN113409382A|2018-10-11|2021-09-17|百度在线网络技术（北京）有限公司|Method and device for measuring damaged area of vehicle|
    CN110147706A|2018-10-24|2019-08-20|腾讯科技（深圳）有限公司|The recognition methods of barrier and device, storage medium, electronic device|
    CN109344804A|2018-10-30|2019-02-15|百度在线网络技术（北京）有限公司|A kind of recognition methods of laser point cloud data, device, equipment and medium|
    CN110047276B|2019-03-11|2020-11-27|广州文远知行科技有限公司|Method and device for determining congestion state of obstacle vehicle and related product|
    CN110111603B|2019-05-10|2021-05-18|广州达泊智能科技有限公司|Parking assisting method, device and system based on three-dimensional detection technology|
    CN110246183A|2019-06-24|2019-09-17|百度在线网络技术（北京）有限公司|Ground contact point detection method, device and storage medium|
    CN110488308A|2019-07-05|2019-11-22|北京国泰新能科技发展有限公司|A kind of compartment position finding and detection method and device|
    KR102363501B1|2019-12-10|2022-02-16|주식회사 라이드플럭스|Method, apparatus and computer program for generating earth surface data from 3-dimensional point cloud data|
    WO2021134258A1|2019-12-30|2021-07-08|深圳元戎启行科技有限公司|Point cloud-based target tracking method and apparatus, computer device and storage medium|
    CN111325984A|2020-03-18|2020-06-23|北京百度网讯科技有限公司|Sample data acquisition method and device and electronic equipment|
    CN111895931B|2020-07-17|2021-11-26|嘉兴泊令科技有限公司|Coal mine operation area calibration method based on computer vision|
    法律状态:
    优先权:
    申请号 | 申请日 | 专利标题
    CN201610057306.XA|CN107025642B|2016-01-27|2016-01-27|Vehicle's contour detection method and device based on point cloud data|
    [返回顶部]